Recently, the performance and functionality of digital cameras and digital movie cameras that use some solid-state image sensor such as a CCD and a CMOS (which will be sometimes simply referred to herein as an “image sensor”) have been enhanced to an astonishing degree. In particular, the size of a pixel structure for use in a solid-state image sensor has been further reduced these days thanks to rapid development of semiconductor device processing technologies, thus getting an even greater number of pixels and drivers integrated together in a solid-state image sensor. As a result, the resolution of an image sensor has lately increased rapidly from around one million pixels to ten million or more pixels in a matter of few years. On top of that, the quality of an image captured has also been improved significantly as well. As for display devices, on the other hand, LCD and plasma displays with a reduced depth now provide high-resolution and high-contrast images, thus realizing high performance without taking up too much space. And such video quality improvement trends are now spreading from 2D images to 3D images. In fact, 3D display devices that achieve high image quality although they require the viewer to wear a pair of polarization glasses have been developed just recently.
As for the 3D image capturing technology, a typical 3D image capture device with a simple arrangement uses an image capturing system with two cameras to capture a right-eye image and a left-eye image. According to the so-called “two-lens image capturing” technique, however, two cameras need to be used, thus increasing not only the overall size of the image capture device but also the manufacturing cost as well. To overcome such a problem, methods for capturing multiple images with parallax (which will be sometimes referred to herein as a “multi-viewpoint image”) by using a single camera have been researched and developed. Such a method is called a “single-lens image capturing method”. For example, Patent Document No. 1 discloses a technique for obtaining two images with parallax at the same time using color filters. FIG. 16 schematically illustrates an image capturing system that adopts such a technique. The image capturing system that uses that technique includes a lens 3, a lens diaphragm 19, a light beam confining plate 20 with two color filters 20a and 20b that have mutually different transmission wavelength ranges, and a photosensitive film 21. In this case, the color filters 20a and 20b may be filters that transmit red- and blue-based light rays, respectively.
In such an arrangement, the incoming light passes through the lens 3, the lens diaphragm 19 and the light beam confining plate 20 and produces an image on the photosensitive film 21. In the meantime, only red- and blue-based light rays are respectively transmitted through the two color filters 20a and 20b of the light beam confining plate 20. As a result, a magenta-based color image is produced on the photosensitive film 21 by the light rays that have been transmitted through the two color filters. In this case, since the color filters 20a and 20b are arranged at mutually different positions, the image produced on the photosensitive film 21 comes to have parallax. Thus, if a photograph is developed with the photosensitive film and viewed with a pair of glasses, in which red and blue films are attached to its right- and left-eye lenses, the viewer can view an image with depth. In this manner, according to the technique disclosed in Patent Document No. 1, multi-viewpoint images can be produced using the two color filters.
According to the technique disclosed in Patent Document No. 1, the light rays are imaged on the photosensitive film, thereby producing multiple images with parallax there. Meanwhile, Patent Document No. 2 discloses a technique for producing images with parallax by transforming incoming light into electrical signals. FIG. 17 schematically illustrates a light beam confining plate 22 according to such a technique. Specifically according to that technique, a light beam confining plate 22, which has a red ray transmitting R area 22R, a green ray transmitting G area 22G and a blue ray transmitting B area 22B, is arranged on a plane that intersects with the optical axis of the imaging optical system at right angles. And by getting the light rays that have been transmitted through those areas received by a color image sensor that has red-, green- and blue-ray-receiving R, G and B pixels, an image is generated based on the light rays that have been transmitted through those areas.
Patent Document No. 3 also discloses a technique for obtaining images with parallax using a similar configuration to the one illustrated in FIG. 17. FIG. 18 schematically illustrates a light beam confining plate 23 as disclosed in Patent Document No. 3. According to that technique, by making the incoming light pass through R, G and B areas 23R, 23G and 23B of the light beam confining plate 23, multiple images with parallax can also be produced.
Patent Document No. 4 also discloses a technique for generating multiple images with parallax using a pair of filters with mutually different colors, which are arranged symmetrically to each other with respect to an optical axis. By using red and blue filters as the pair of filters, an R pixel that senses a red ray observes the light that has been transmitted through the red filter, while a B pixel that senses a blue ray observes the light that has been transmitted through the blue filter. Since the red and blue filters are arranged at two different positions, the light received by the R pixel and the light received by the B pixel have come from mutually different directions. Consequently, the image observed by the R pixel and the image observed by the B pixel are ones viewed from two different viewpoints. By defining corresponding points between those images on a pixel-by-pixel basis, the magnitude of parallax can be calculated. And based on the magnitude of parallax calculated and information about the focal length of the camera, the distance from the camera to the subject can be obtained.
Patent Document No. 5 discloses a technique for obtaining information about a subject distance based on two images that have been generated using either a diaphragm to which two color filters with mutually different aperture sizes (e.g., red and blue color filters) are attached or a diaphragm to which two color filters in two different colors are attached horizontally symmetrically with respect to the optical axis. According to such a technique, if light rays that have been transmitted through the red and blue color filters with mutually different aperture sizes are observed, the degrees of blur observed vary from one color to another. That is why the degrees of blur of the two images that are associated with the red and blue color filters vary according to the subject distance. By defining corresponding points with respect to those images and comparing their degrees of blur to each other, information about the distance from the camera to the subject can be obtained. On the other hand, if light rays that have been transmitted through two color filters in two different colors that are attached horizontally symmetrically with respect to the optical axis are observed, the direction from which the light observed has come changes from one color to another. As a result, two images that are associated with the red and blue color filters become images with parallax. And by defining corresponding points with respect to those images and calculating the distance between those corresponding points, information about the distance from the camera to the subject can be obtained.
According to the techniques disclosed in Patent Documents Nos. 1 to 5 mentioned above, images with parallax can be produced by arranging RGB color filters on a light beam confining plate or a diaphragm. However, since the RGB based color filters are used, the percentage of the incoming light that can be used decreases significantly. In addition, to increase the effect of parallax, those color filters should be arranged at distant positions and should have decreased areas. In that case, however, the percentage of the incoming light that can be used further decreases.
Unlike these techniques, Patent Document No. 6 discloses a technique for obtaining multiple images with parallax and a normal image that is free from the light quantity problem by using a diaphragm in which RGB color filters are arranged. According to that technique, when the diaphragm is closed, only the light rays that have been transmitted through the RGB color filters are received. On the other hand, when the diaphragm is opened, the RGB color filter areas are outside of the optical path, and therefore, the incoming light can be received entirely. Consequently, multi-viewpoint images can be obtained when the diaphragm is closed and a normal image that uses the incoming light highly efficiently can be obtained when the diaphragm is opened.